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PREFACE 

This workbook is the primary training tool of a special 
project, funded by the State of Ohio and operated by Youngs- 
town State University, the purpose of which is to aid United 
Appeal agencies in the Youngstown area in improving their 
planning and budgeting processes. It is designed to make the 
concept of program budgeting available to smaller social agen- 
cies in a simplified and practical forme 

The workbook is structured so that when it is completed, 
the agency member should have a complete record of the process 
used to formulate future programs for the agency* The workbook 
provides a process; for the agency to reach a working consensus 
about goals and priorities; it gives instruction and practical 
exercise in writing specific objectives; it aids the agency in 
developing a program structure and budget; it provides a pro- 
cess for immediately assessing agency performance levels; and 
suggests the steps necessary in establishing a formal evalua- 
tion system. 

The creation of this workbook is due to the direct and 
indirect efforts of many people. Those who were instrumental 
in planning the project anC. obtaining funding for it are Dr. 
Jack Foster and Dr. Lawrence Looby. Their efforts, along with 
Mr. William Brennan and Mr. Joseph Gray of the Youngstown 

Community Chest, resulted in the acceptance of this project 
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by local United Way Agencies. 

The individuals printarily responsible for the operation 
of the project and for the development of this workbook and 
the accompanying media presentation include Dr. Steve Redburn, 
Mr. Frank Huntley, Mr. Mark Cohen, D. Edgar Cobett, and Mr* 
Kelvin Carlisle. Mr. Steve Grcevich is the announcer. 

Two advisory committees assisted in reviewing and comment- 
ing on the text of this workbook. The YSU Faculty Advisory 
Committee consisted of Ms. Syretha Cooper, Dr. John Kearns and 
Dr. Raymond Shuster. Ttie Community Chest Advisory Committee 
consisted of Mr. Raymond Brenner, Jr., Mr. Harold Halls, Ms. 
Carol Harding, Mr. Gregg Hunger ford, Ms. Diane Schumacher, and 
Rev. David Stone. 

Clerical and secretarial help included Ms. Claire Strow, 
Ms. Mary Willmitch, Ms. Debby Wilson, Ms. Lynn Zigoris, and 
Sheri Zubal. 

A special note should go to those organizations whose 
materials were used as a basis for sections of this workbook. 
These include Phi Delta Kappa, United Way of America, and the 
Urban Institute. 
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INTRODUCTION 

The major goal of the Project to Improve Social Agency 
Planning and Budgeting is the adoption by each Community 
Corporation-supported agency of a new approach to decision- 
making. This approach differs from the current practice of 
most agencies in the degree to which it: 

1. Makes the goals of the agency explicit and visible 
to both board and staff. 

2, Establishes measurable, short-term objectives against 
which to measure agency performance. 

3, Organizes the budget so as to make clear to all the 
dollar cost of achieving (or failing to achieve) 
each objective. 

4. Establishes methods for measuring scientifically the 
degree to which the agency achieves its objectives, 
other impacts of the agency programs, and the needs 
for service c£ target populations or potential clien- 
teles • 

5« Permits annual reconsideration of goals and objectives 
in light of a precise benefit/cost analysis of the 
previous year's performance as well as reassessment* 
of target populations? and 

6. Relates annual budget decisions to precise benefit/ 

cost analysis of the previous year's performance 
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and estimated benefit/cost ratios of as yet 




untried alternative programs. 



A fully developed Planning-Progr aiming-Budgeting system 
ties all of these elements together in an annual decision- 
making cycle so that evaluation is relevant to planning^ 
planning is tied to budgeting, and budgeting is keyed to 
measures of actual performance and information about the 
agency's changing environment. This cycle is illustrated 



below: 



Figure 1: 
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Eleinent 5 of the cycle is an administrative function 
and is not discussed here. All of the other elements are 
part of the agency's decision-making process and will ordin- 
arily involve both board and staff. Needs assessment/ which 
is the process of collecting data abci.t the agency's environ- 
ment or target population is a key imput to the goal setting 
process but will not be dealt with in much detail for lack 
of space. It is discussed briefly in Chapter Five. , 

The firs four workbook chapters correspond to the 
elements of the annual program budgeting cycle as follows: 
Chapter One: Goal Setting 

Chapter Two: Defining Measurable Objectives 

Chapter Three: Programming and Budgeting 
Chapter Four: Program Evaluation 

Chapter Five is intended to provide guidance to the 

agency in implementing a fully-integrated program budgeting 

process** 

The authors of this workbook modestly ask you to remember 
two things: (1) the conception of program budgeting offered 
here is a drastically reduced and simplified one, designed for 
agencies with limited resources and expertise; and (2) the 
transformation of an agency's annual decision-making process 
called for here will take time to implement fully and will 
require the abandoning of many old, comfortable habits. For 
instance, the collection of accurate information on program 
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effectiveness may require one to discard some cherised preju- 



dices. 

The payoffs of program budgeting, which will not appear 
immediately, include : 

1. improved program effectiveness 

2. greater efficiency in the use of scarce resources 

3. increased ability to justify agency programs when 
seeking funds 

4. improved staff morale 

5. a more adaptable organization, more likely to 

survive and better able to match its efforts 

with the needs of the community, 

*In actual practice the various elements of the annual, 
decision-making cycle are not as clearly separated as suggested 
in Figure 1, For instance, prograirxning (deciding what to do) 
depends not only on objectives but also on how much money is 
to be available, i.e., budgeting. Also, an objective that is 
consistent with the agency's goals may have to be dropped 
either because the agency finds it lacks capability or for rea- 
sons of dost; in this way programming and budgeting may force 
a redefinining of objectives. So Figure 1 represents only 
roughly the time sequence of annual decision-making by board 
and staff. 
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GLOSSARY 



accountability - synonym for responsibility? implies that officials 

will report, explain, and/or justify all activities^ 

activity - identifiable action that is conducted in order to 
achieve a specific objective • 

budget - written plans for allocating financial resources • 

costs - specific resources (inputs) expressed in monetary terms 
required to achieve a given output. 

criteria - standards by which something is judged or evaluated. 

•effectiveness - a measure of the degree to which an objective 

is attained. 

efficiency - minimum cost at which a specified level of impact 
or outcome can be maintained or maximum output for 
a specified cost level . 

element - aggregations of activities directed toward a particular 
objective. 

evaluation criteria - standards by which to' determine program 

effectiveness (impact) . 

goal - statement of broad direction, purpose, or intent based 
on identified needs. 

input - resources or activities that are used to achieve an 
objective. 

objective - a measurable outcome of a specific activity, element 
or program. 

outcome - impact or change in the environment that occurs when 
resources (inputs) are used to achieve a specific 
objective. 

output - services provided 

planning - process of deciding on the objectives, strategies, 
and resources to be created and used in fulfilling 
an agency * s purpose . 

policies - written statements which express the purpose, inten- 
tions and conditions under which an .agency operates. 



ERIC 10 



9 



program - group of interdependent, closely related elements and 
activities contributing to a common objective. 

program budget - a written plan for allocating financial resources 

among program categories based upon outputs. 

program structure - hierarchical arrangement of programs which 

represents the relationships of activities^ 
elements and programs to specific objectives 
or outputs. 
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CHAPTER ONE 
EXAMINING AGENCY GOALS AND PRIORITIES 
Overview 

Goal-setting is simply an agency's internal political 
process. It goes on in every organization all of the time. 
Here we are interested in making goals explicit and deter- 
mining to what degree a consensus exists within the agency 
over v*iat its purposes are. 

In the absence of consensus, making goals explicit will 
increase conflict. This is not necessarily bad for the or- 
ganization provided there is a set of "decision rules," more 
or less accepted by everyone, for controlling debate and 
eventually reaching a decision. One of our premises in seek- 
ing to make goals explicit is that creative conflict , conducted 
within a framework of decision rules, is healthy for the 
organization. 

Conflict that is rationally structured and limited by a 
spirit of mutual tolerance can accomplish several things. 
First, it is likely to bring to the surface and illuminate the 
two sets of factors which lead individuals to adopt different 
goals. These are: (1) differences in values and (2) differ- 
ences in perceptions (of the nature of the--problems being 
addressed, of the utility of various means of dealing with 
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these problems, and of the capabilities of the agency and 
the individuals within it). Where goal conflict results 
primarily from differences in perceptions # the availability 
of new information can eliminate many apparently deep divis- 
ions. This information can be generated by a properly designed 
planning process that includes both needs assessment and 
program evaluation > So one effect of ir.aking goals explicit 
should be to highlight the need for this new information. 

Secondly, even where conflicts result from differences 
in values, frank and rational discussion can actually lead to 
a new consf^nsus. This can occur only when the conflicting 
individuals are adept at listening and at adopting the per- 
spectives of their opponents. The development of such skills 
is beyond the scope of this training process. However, there 
are a number of widely-used methods available to organizations 
that want to work through strains of this sort. *<See note-page 

Even in organizations that already possess a goal con- 
sensus, there is value in precisely stating the nature of 
this consensus. A precisely stated set of goals provides the 
basis for constructing measurable short-term objectives (see 
Chapter Two) vAiich in turn enables the agency to measure its 
performance relative to these objectives and to feed this in- 
formation into the budgetary process. In other words, an 
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explicit statement of goals sets the keystone of the agency's 
program structure and constitutes the first element in an 
integrated planning-programming-budgeting cycle. 

This chapter contains two alternative procedures for 
establishing an agency's goals. In each case, the mechanical 
procedures employed are rather arbitrary devices for bringing 
about a rationally-structured discussion of goals within the 
very limited time available. The emphasis should not be on 
the procedure but on a creative discussion that will clarify: 
(1) the degree to v^ich there is concensus or conflict over 
goals on the board and between board and staff; (2) the sources 
of conflict (differences in values and differences in percep- 
tion) ; and (3) the agency's needs for new information about 
its environment (needs assessment) and about its performance 
(program evaluation) that would help to resolve conflicts or 
to generate new goals for the organization. 

The Basic Procedure, v*iich begins on page 14, is 
designed for agencies with more diverse functions or agencies 

*Two useful guides to these organizational development 
methods are: Fordyce^ Jack K. and Raymond Weil, Managing 
WITH People # A Manager's Handbook of Organization Development 
Methods, Addison-Wesley (1971) ; French, Wendell L. and Cecil 
H. Bell, Jr. , Organization Development, Behavioral Science 
Interventions for Organization Improvement , Prentice-Hall (1973) 
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that anticipate significant conflict over goals. The Alter- 
native Procedure* beginning on page 25, has been developed 
for use by agencies that believe they possess a good consensus 
or agencies that perform a very limited range of functions.. 

SELECTED REFERENCES 



Hughes* Charles L., Goal-Setting: Key to Individual 
and Organizational Ef fectiveness o American Management 
Association, 1965. 
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Directions for Individuar""Rankinq of Goal Statements 

1. Make sure your materials include the following items: 
a. One (1) Display Board listing 23 goal statements 

A set of thirty (30) red squares 

2. Read each of the Goal Statement Cards, As you read each 
card/ ask yourself: 

"HOW IMPORTANT IS THIS GOAL STATEMENT FOR OUR AGENCY?" 
3* Read each goal statement. For every goal statement you 

believe to be important to your agency, place a red square 
beside it in the column labeled #1. 

4. Read the goal statements that have one square beside them. 
For those goal statements you believe to be more important 
than others, place a red square beside them in the column 
labeled #2. 

5. Read the goal statements that have two red squares beside 
them. For those goal statements you believe to be much more 
important than others, place a red square beside them in 
the column labeled #3. 

6. Read the goal statements which have three squares beside 
them. For those goal statements you believe to be of great 
importance, place a red square beside them in the column 

. labeled #4. 

7. Read the goal statements which have four red squares beside 
them. For those goal statements you believe to be of extreme 

16" 
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importance^ place a red square beside them in the 
column labeled #5. 

8. Review your Display Board and keep in mind the following: 

a. All 30 red squares must be used. (Each square has 
a value of 1 point) • But, you may have more than 
one goal statement with five red squares beside it. 

b. At least one goal statement must have five red 
squares (5 points) beside it. 

c. A maximiom of five red squares (5 points) is allowed 
for any one goal statement. 

d. It is not necessary for a goal statement to have a 
red square beside it. 

e. In the event you wish to rearrange your display 
board/ you may add or remove red squares (points) 
from the goal statements (remembering that squares 
must always be in horizontal sequence with no spaces 
between squares) • 

9. Transfer the total number of points for each goal statement 
to the Individual Goal Statement Summary Sheet. (It is 
important to note that the goal statements found on your 
Individual Goal Statement Summary Sheet are in random 
order and will not match the order in vAiich you placed 
your goal statements) . 
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10, You will be assigned to a small group (3 5 persons). 
After a break, you will begin working with your group 
on arriving at a consensus on a single display board. 

11. .Leave your, display board at your position. Take your 
workbook to your small group meeting. 
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INDIVIDUAL GOAL RATING SHEET 
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Instructions: Place the total number of points (red squares) you gave to each 
of the goals on your Display Board in the blank space found 
next to each of the goals on this page, 

OUR GOALS My Individual 

Score for Each Rank Order 
Goal 

1. Create opportunities for full employment of all 

those who can be gainfully employed. 

2. Provide enough income to meet the basic needs 

of those unable to provide for themselves. 

3. Protect consumers from fraud, unfair trade 

practices, and unsafe products. 

4. Provide food and nutrition guidance to those 

who can't provide these for themselves. 

5. Provide clothing to those in need of 
clothing and unable to obtain it for 
themselves . 



6. Provide assistance to individuals, families, 
and the community to insure adequate 
housing and fair housing practices. 

7. Provide fast, safe, efficient and convenient 
means of transporting individuals and 
goods from one place to another. 



8. Provide individuals with environments where 
they feel safe and are protected from anti- 
social and criminal elements. 



9. Promote justice, equal protection of the 
law and the means for resolving disputes 
without force. 



10. Protect and enrich the quality of the 
non-human environment: air, water, animal 
life, etc. 

11. Foster physical health and assure 
adequate care and treatment for illness 
and disability. 

12. Foster mental health and assure adequate 
care and treatment for mental illness and 
disability . 

13. Help' those with inadequately developed 
intelligence to learn and to adapt to the 
demands of society. 




14. Help the physically disabled overcome 
handicaps and realize their maximum 
potential. 



15. Help individuals acquire the knowledge and 
skills provided by schools and colleges. 

16. Help people acquire knowledge and skills 
outside the framework of the formal 
educational sys tem . 

17. Preserve, strengthen, and (where necessary) 
substitute for the family. 

18 . Promote the individual 's sound personality 
development and social maturity. 

19. Promote the arts and humanities, in order 
to enrich the cultural and spiritual 
development of the individual. 

20. Encourage participation of individuals in 
democratic processes and social activities 
aimed at improving their lives and those 
of others. 



21. Promote community economic development. 

22. Provide assistance to other human services 
organizations in achieving goals which they 

have identified as important. 

23. Promote equality of opportunity and assist 
victims of discrimination on the basis of 
age, sex, race, color, creed, national 

origin, or income level. 

Remember you will need this sheet for your small group meeting! 
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Directions for Small Group Ranking of Goal Statements 

1. The small group materials should include:. 

a. One (1) Display Board containing goal statements 

b. A set of thirty (30) red squares 

c. Chart of Averages 

d. Small Group Goal Statement Summary Sheet for each 
individual 

2. There should be 3 to 5 members assigned to each group. 

3. Each group will select someone to act as group leader and 
tally the scores. 

Round 1 ; Directions for Group Leader 

4. Read the first goal statement, and have each member read 
aloud his score for that particular goal statement. 

5. Add all the scores for that goal statement and consult the 
"Chart of Averages" to determine the average score for that 

particular goal. 

6. Record the average for that goal statement in the average 
column of the Small Group Goal Statement Summary Sheet. 

7. Repeat steps 4 through 6 for each goal statement. 
Round 2 ; 

8. Using the mathematical averages as an initial guide, the 
group is to arrive at a consensus score for each goal state- 
ment. (A consensus score is reached when 2 out of 2, 3 out 
of 4, or 4 out of 5 members of the group agree upon a score.) 

ERIC ^ 
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9. The consensus score is indicated on the Display Board by 
placing the appropriate nuinber of squares next to that 
goal statement. 

10. If a consensus score cannot be reached, the row next to the 
goal statement is left blank and the next goal statement 

is discussed. 

11. It is possible: 

a. That all red squares (30 points) will be used before 
the group has ranked all of the twenty-three (23) goal 
statements. In this case, the group will have to read 
through the goal statements and make consensus adjust- 
ments as required ( as you did as individuals ) . 

b. That all of the red squares will not be used in the 
initial group ranking process. In this case, the 
group will have to read through the goal statements and 
make consensus adjustments because all red squares must 
be used. 

c. That a final consensus cannot be reached on one or 
more goal statements. If this should occur, please in- 
form the coordinator. 

12. Record the consensus score for each goal statement in the 
consensus column of the Small Group Goal Statement Summary 
Sheet. 
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13. Notify the coordinator that your group has completed its 
task, 

14, Re-assemble as a total group to discuss the small group 
ratings and reach a total group consensus for each goal 
statement. 
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SMALL GROUP GOAL STATEMENT SUMMARY SHEET 



GOAL STATEMENTS Members Average Consensus Rank 

Scores Score Score 1^ 

1. Create opportunities for full employment 

of all those who can be gainfully employed. 



2. Provide enough income to meet the basic 
needs of those unable to provide for 
themselves . 



3. Protect consumers from fraud, unfair 
trade practices, and unsafe products. 

4. Provide food and nutrition guidance to 
those who can't provide these for 
themselves . 



5. Provide clothing to those in need of 
clothing and unable to obtain it for 
themselves . 



6. Provide assistance to individuals, 
families , and the community to insitre 
adequate housing and fair housing 
practices . 

7. Provide fast, safe, efficient and 
convenient means of transporting 
individuals and goods from one place 
to another. 



8. Provide individuals with environments 
where they feel safe and are protected 
from anti-social and criminal elements. 



9. Promote justice, equal protection of 
the law and the means for resolving 
disputes without force . 

10. Protect and enrich the quality of the 
non-human environment: air, water, 
animal life, etc. 

11. Foster physical health and assure 
adequate care and treatment for 
illness and disability. 

12. Foster mental health and assure adequate^ 
care and treatment for mental illness 
and disability. 



ERIC 



24 



13, Help those with inadequately developed 
intelligence to learn and to adapt to 
the demands of society. 

14. Help the physically disabled overcome 
handicaps and realize their maximum 
potential. 

15- Help individuals acquire the knowledge and 
skills provided by schools and colleges , 

16. Help people acquire knowledge and skills 
outside the framework of the formal 
educational system. 

17. Preserve, strengthen, and (where 
necessary) substitute for the family. 

18. Promote the individual's sound 
personality development and social 
maturity . 

19. Promote the arts and humanities, in 
order to enrich the cultural and 
spiritual development of the 
individual . 

20. Encourage participation of individuals 
in democratic processes and social 
activities aimed at improving their 
lives and those of others. 

21. Promote community economic develop- 
ment . 

22. Provide assistance to other human 
services organizations in achieving 
goals which they have identified as 
important . 

23. Promote equality of opportunity and assist 
victims of discrimination on the basis 

of age, sex, race, color, creed, national 
origin, or income level. 
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CHART OF AVERAGES 



Total 

Score Group of 3 Group of 4 Group of 5 



0 


0 


0 


0 


1 


.33 


.25 


.20 


2 


.67 


.50 


.40 


3 


1 .00 


.75 


.60 


4 


1.33 


1 .00 


.80 


5 


1.67 


1 .25 


1.00 


6 


2.00 


1.50 


1 .20 


7 


2.33 


1.75 


1.40 


8 


2.67 


2.00 


1.60 


9 


3.00 


2.25 


1.80 


10 


3.33 


2.50 


2.00 


11 


3.67 


2.75 


2.20 


12 


4.00 


3.00 


2.40 


13 


4.33 


3.25 


2.60 


14 


4.67 


3.50 


2.80 


15 


5.00 


3.75 


3.00 


16 




4.00 


3.20 


17 




4.25 


3.40 


18 




4.50 


3.60 


19 




4.75 


3.80 


20 




5.00 


4.00 


21 






4.20 


22 






4.40 


23 






4.60 


24 






4.80 


25 






5.00 
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SMALL GROUP MEETINGS 



SUMMARY SHEET TOTAL 



OUR GOALS 



A B C D E F 



H Tot. Avg. Rank S 



2* 



Create opportunities for full 
employment of all those who can 
be gainfully employed. 



2. Provide ^mough income to meet the 
basic UcJ^ds of those unable to 
provide i'or themselves. 



3. Protect consumers from fraud, 
unfair trade practices, and 
unsafe products. 



Provide food and nutrition 
guidance to those who can't 
provide these for themselves . 



Provide clothing to those in 
need of clothing and unable 
to obtain it for themselves . 



Provide assistance to 
individuals, families, and the 
community to insure adequate 
housing and fair housing 
practices . 



Provide fast, safe, efficient 
and convenient means of trans- 
porting individuals and goods 
from one place to another. 



provide individuals with 
environments where they feel safe 
and are protected from anti-soclai 
and criminal elements . 



9. Promote justice, equal 

protection of the law and the 
means for resolving disputes 
without force. 



10. Protect and enrich the quality 
of the non~human environment: 
air, water, animal life," etc. 



11. Foster physical health and 
ppj^assure adequate care and treat- 
^J^^aent for illness and disability. 
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12, Foster mental health and assure 
adequate care and treatment for 
mental illness and disability. 



13, Help those with inadequately 
developed intelligence to 
learn and to adapt to the 
demands of society. 

14, Help the physically disabled 
overcome handicaps and 
realize their maximum 
potential • 



15, Help individuals acquire 
the knowledge and skills 
provided by schools and 
colleges . 



16, Help people acquire knowledge 
and skills outside the 
framework of the formal 
educational system. 



17, Preserve, strengthen, 
and (where necessary) 
substitute for the family. 



18. Promote the individual's 

sound personality develop- 
ment and social maturity. 



19. Promote the arts and humani- 
ties, in order to enrich the 
cultural and spiritual 
development of the 
individual . 



20. Encourage participation of 
individuals in democratic 
processes and social activi- 
ties aimed at improving their 
lives and those of others . 



21. Promote community economic 
development . 

22. Provide assistance to other 
human services organizations 
in achieveing goals which they 
have identified as important. 
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23.. Promote equality of oppor- 
tunity and assist victims 
of discrimination on the 
basis of age, sex, race, 
color, creed, national 
origin, or income level. 



S equals Estimate of Population Variance 
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LARGE GROUP PRIORITY RANKING OF GOALS 



OUR GOALS Average Rank 

Score Order 



1. Create opportunities for full employment of all 
those who can be gainfully employed. 

2. Provide enough income to meet the basic needs 
of those unable to provide for themselves. 



3. Protect consumers from fraud, unfair trade 
practices, and unsafe products. 



4. Provide food and nutrition guidance to those 
who can't provide these for themselves. 



5. Provide clothing to those in need of 
clothing and unable to obtain it for 
themselves . 



6. Provide assistance to individuals, families, 
and the community to insure adequate 
housing and fair housing practices. 



7. Provide fast, safe, efficient and convenient 
means of transporting individuals and 
goods from one place to another. 

8. Provide individuals with environments where 
they feel safe and are protected from anti- 
social and criminal elements. 



9. Promote justice, equal protection of the 
law and the means for resolving disputes 
without force . 



10. Protect and enrich the quality of the 

non-human environment: air, water, animal 
life , etc . 



11. Foster physical health and assure 
adequate care and treatment for illness 
and disability. 

12. Foster mental health and assure adequate 
care and treatment for mental illness and 
disability • 

13. Help those with inadequately developed 
intelligence to learn and to adapt to the 
demands of society. 
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14. Help the physically disabled overcome 
handicaps and realize their maximum 
potential • 

15. Help individuals acquire the knowledge and 
skills provided by schools and colleges • 

16. Help people acquire knowledge and skills 
outside the framework of the formal 
educational system. 

17. Preserve, strengthen, and (where necessary) 
substitute for the family. 

18. Promote the individual's sound personality 
development and social maturity. 

19. Promote the arts and humanities, in order 
to enrich the cultural and spiritual 
development of the individual. 

20. Encourage participation of individuals in 
democratic processes and social activities 
aimed at improving their lives and those 
of others. 

21. Promote community economic development. 

22. Provide assi^itance to other human services 
organizations in achieving goals which they 
have identified as important. 

23. Promote equality of opportunity and assist 
victims of discrimination on the basis of 
age, sex, race, color, creed, national 
origin, or income level. 



o 
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Alternative Procedure for 
Examining Agency Goals and Priorities 

Some agencies^ because of the narrow scope of their 
functions or because they already possess a rough consensus 
as to goal priorities, may find it more productive to employ 
an alternate goal-setting procedure. This alternative assumes 
a general consensus and seeks to accomplish two things: (1) 
to make explicit the already existing consensus; (2) to dif- ■ 
ferentiate between v*iat is actually known concerning the 
agency's environment and what is simply assumed. 

Procedure 

1. Examine the list of general goal statements on 
page 17 in the workbook and select one or two 
statements that you believe are most like the goals 
of your agency. 

2. The coordinator will assist in determining, probably 
by a show of hands, to v*iat extent a consensus exists 
on these choices, 

3. Anyone may volunteer a suggested rewording of these 
goals to make them actual statements of the agency's 
goals, 

4. The group will discui'M/ the wording of the goal(s) 
and its implications, 

5. Once there is a consensus on the goal statement ( s) , 
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the group should consider the following questions, dif- 
ferentiating for each factual statement between that v*iich 
is known and that v^iich is assumed : 

a« What are the local dimensions of the 

problem this agency is addressing (e.g., 

how many people, where located, how affected)? 

b. What are the known and suspected causes of 
the problem? 

c. At present, how capable is the agency of 
dealing with the problem (e.g., what pro- 
portion of the tcirget population can be 
reached and helped, v^at limits are imposed 
on the agency's performance, by lack of funds, 
personnel, expertise, or political support)? 

&• In light of answers to the previous questions, the group 
should tentatively list its needs for information about 
(1) the needs of its target population and (2) the reach 
and impact of its programs relative to that population. 
You have just successfully completed phase one of your 

mission. 

You have had the opportunity to (1) make clear to your- 
self your own goal priorities and (2) compare your priorities 
with those of others. 



You have achieved, we hope, a preliminary consensus on 
the top priorities of your agency# 

Now, these goals will become the basis for reorgan- 
izing the budget of your agency» 

In the next session, you will develop measurable short- 
term objectives under each of the goals you have listed as 
high in priority. You can begin thinking now about how you 
would determine whether your agency is making progress toward 
each goal* 

In the third session, with the assistance of your pro- 
fessional staff, you will transform the current budget of 
your agency into a pr ogr am/budget vAiich will show you how 
much it is costing you to move toward each of your objectives* 

In the fourth session, you will be introduced to the 
methods of program/evaluation and asked to do two things: 
(1) discuss how you can best measure the impact your agency 
is having relative to each objective and (2) compare with 
each other your own current assessments of agency performance 
relative to each objective. 

At the end of these four sessions you can then begin 
to implement the outgoing activities that contribute to suc- 
cessful program budgeting s- successful because it leads to 
improved agency effectiveness. 

Thank You for your contribution to the successful com- 
pletion of this first phase of your mission. 
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Chapter Two 
DEFINING MEASURABLE OBJECTIVES 
Overview 

Before goal-setting can lead to improved organiza- 
tional effectiveness, goals must be translated into more 
specific statements of intention that (1) identify specific 
results expected, (2) are quantified and measurable, and (3) 
state a precise time frame for performance. The time frame 
most appropriate to objectives linked to a program budget- 
ing system is the budget year. 

For each goal, each year, one or more measurable 
objectives should be identified in advance and used as a 
basis for organizing the budget (see chapter three) . For 
instance, the goal of reducing age discrimination in employ- 
ment might, in a given year, be reduced to such realistic 
objectives as: 

a. eliminating employers' automatic retirement rules 
affecting 500 workers over age 65. 

b. locating part-time employment for 45 retired workers 

c. securing state legislation providing. effective 
penalties for age discrimination in employment. 

Note that objectives state the result to be accomplished 
not the means to be used or the services to be provided. 
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This is very important, since it leaves the way open to 
systematic analysis of alternative means to the same ends. 
Annual consideration of alternatives to present programs 
is an important component of a full-fledged program budget- 
ing system. 

The setting of specific objectives has a number of 
beneficial effects on the organization: 

1. It provides a. clarifying framework for discussion. 
Rhetoric and metaphor are replaced by precise state- 
ments of intention. 

2. It provides a sense of direction and a set of 
realistic targets for all members of the agency. 
There can be a drastic change in the psychology of 
the staff. There is less likely to be need for 
close supervision of lower level employees. In 
fact# the use of objectives in this way has been 
developed into a complex system of management com- 
monly referred to as Management By Objectives or MBO. 

- 3. It provides measurable standards against which to 

judge performance. This is a precondition for scien- 
tific program evaluation (discussed in Chapter Four) . 
4. As noted previously* it allows for systematic analysis 
of alternative means to these ends* and finally 
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5. It enables a restructuring of the agency's budget 
so that the dollar cost of achieving. Or failing to 
reach, each objective is made visible (see Chapter 
Three) . 

SUGGESTED REFERENCES 

Morrisey, George L. Management By Objectives and Results > 
Addison-Wesley, 1970. 

Raia, Anthony P. Management By Objectives , 
Scott, Foresman and Co., 1974. 

McConkey, D.D. "Writing Measurable Objectives for Staff 
Managers." Advancfed Management Journal 37 (January 1972 
10-16) . 
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What is the difference between a goal and an objective? 

Goals are broad, general statements of the results we desire. As general 
statements, they can be interpreted in many different ways. 

Objectives are much more specific than goals. A good objective leaves 
•ittle doubt about what we specifically want done, and when it will be 
accomplished. 



Is the following statement a goal or an objective? 
"Every child shall learn to read." 
Answer: 
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If you said goal , you were right. It is a general statement with many 
interpretations. 

If you said objective , why were you wrong? Does the statement tell us 
specifically what kind of reading performance the child will have? Not really 
just that he will learn to read. Does the statement tell you when the child 
v/ill learn to read? No again. You see how this statement does not specify 
exact outcome? 



Now try this one; 

Is the following a goal or objectiv e? 
*'To enable every first grade child to learn to read a book." 
Answer: 
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If you said goal , you were right. Once again, the statement is too 
general to be a good objective. 



If you said objective , take another look at the statement. Does it tell 
you when the child will learn to read, or what kind of book? 



Once more, is the following statement a goal or objective? 

"Each ten-year-old child will be able to read at least 100 fifth grade 
level words per minute with a 70% retention rate within one year," 



Answer: 
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If you said objective , you were right. You noticed that the statement 
clearly specified which children (ten-year-olds), how well they would read 
(100 fifth grade level words per minute with a 70% retention rate), and 
when (within cne year). 



You may have noticed the qualities of a good objective now. A useful 
objective states a specific result in measurable terms, and tells when the 
result will take place. 



41 
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Now, see if you can list the three qualities of a useful objective. 



1. 

2. 
3. 



Check the follovnng page to see if you know all three. 
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The three qualities of a useful objective are: 

1 . States specifically the result intended . 

2. Is measurable . 

3. Includes a time frame: tells us when the result will happen. 



% 
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Let's spend a few moments thinking about the first two 
qualities of a useful objective which are: 

1. States a specific result in 

2. measurable terms. 

Look at the following statements and put an X in the 
box beside each statement that has these two q ualities. 

1.1 — 1 Enable 25 men to obtain full-time jobs paying at 




least $2.00 an hour. 




maximum feasible participation. 



3-P 



Provide educational opportunity. 




quorum present at the next meeting. 




women into the family planning program. 



6. 




Provide an adequate diet. 



7 




Type 40 words per minute. 




9. 



□ 



Obtain $1,000 in donations. 



10 




Involve the poor. 
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You should have placed an X before statements 1,4,5,7,9. 



Take another look at any you missed. 

Do they tell you specifically how much participation, 
what kind of opportunity or diet, or what kind of involvement? 

Some statements are easy to measure? others can't be 
measured. 

Read the following statements ; 

1. As many as possible. 

2. The maximum amount. 

3. A feasible number. 

Can these statements be measured? 



Answer: 



39 

If you said no , you are right. 

None of the statements can be measured because we are not 
told how many are possible; it could be 5 or 5,000, Nor do 
we know how many are maximum, or feasible; it could be any 
number. If such statements are included in an objective, then 
the objective will not be specific enough to be useful. 

The third quality of a useful objective was that it 
included a time frame. That is, it tells us when we can 
expect the result to happen, or by what date the result will 
be completely accomplished. 

Which of the following statements specify a good time 
frame? 

1 . As soon as possible. 

2. When we are ready. 

3. By July 1, 1977. 

An swer : 
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Did you think that only Number 3 specifically said by 
what time we could expect a result to happen? That's right. 

Statements 1 and 2 don't tell us how soon is possible 
or when we are ready. Is it tomorrow or next year? We 
don ' t know. 

Useful objectives must specify when a result will 
happen by stating a date or giving a number of days, months , 
or years. 



Just to be sure you have it, list again the three 
qualities of a good objective. 

1. - 

2 . • — 

3. ■ 

If you are not sure, check your answers on page 36 • 
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Now let's see if you can find these three qualities in an objective. 



Objective: To enable twenty people to get full-time jobs paying 
. at least $2.00 per hour by July 1, 1977. 

1. Underline the parts of the objective above that state the 
intended result. 



2. Place a |square| around the parts that are measurable. 



3. (Circle J the time frame. 
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The answers are: 



To enable ftiwg^ntvj people to get If\:ill-tinigM obs paying at least 



($2,00 per hour ] by( ^July 1, 1977. 



Now do the same with the following objectives ; 

1. Underline the specific result intended 

2. Put a fsguar'el around the measurable parts. 

3. (^Circl^) the time frame. 

1. To enable 100 poor people , to get chest x-rays by July 1, 
1977. 

2. To place 100 poor people in full-time jobs within one 
year paying at least $2.00 per hour. 

3. To enable 50 drop-outs to return to high school by 
September 15, 1977. 
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The answers are ; 
1. To enable LlOol poor people to get chest x-rays by (julyJL. 



2. To place f 1,00/ poor people in ! full-time] jobs (w ithin one 
"year) paying at ^ f>7=i^f'J$2 .00 per hourl 

3. To enable >5Q/ drop-outs to return to high school by 



(^eptember 15, 19T7J) 



Place an X before each statement below that you would 
accept as meeting the three qualities of a usefu l objective; 

1, I j to identify 75 dropouts and enable them to return to 

high school by September 15. 

2, I 1 to provide one hour of tutoring in basic mathematics 

to 25 ninth-graders, three nights a week from October 1 
to May 15. 

3 , |- — I to provide all low-income twelfth graders with a list 

of financial aid programs for college students by 
December 1. 

4, 1^ — j to create a special class for high - achieving ninth- 

graders who are low-income by January 15. 

5, j — j to encourage the school board to provide free lunches 

for all low-income students during the next. school yeac 
6, I I to enable 30 non-high-school graduates to take and pass 
the twelfth-grade equivalency test by July 1. 
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Ansv/ers : 

You should have placed an X before 1, 2, 3, 5. 

Take another look at any you missed, and see if you can determine 
why you made the mistake. 

Number 4 doesn't tell us what the special class is for or how many 
ninth-graders will participate. It is not specific or measurable. 

Number 5 doesn't tell us what the specific result will be. It only 
says that the school board will be "encouraged." 

Remember that an objective can qualify as a measurable objective and 
still not contribute to the accomplishment of a goal. Some of the measur- 
able objectives listed on the preceeding page would fall into that cate- 
gory. For example, Numoer 2 may be measurable, but it does not relate 
directly to the goal of "upgrading the education level of teenage 
dropouts." 
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You should be careful that the **what to do" does not become the 
"how to do it." 

For example, the objective "to enable 100 poor people to get chest 
x-rays by July 1, 1977" tells us what needs to be accomplished, but it 
does not restrict how we might accomplish it. We could provide the 
equipment and personnel and give the x-rays ourselves; we could contract 
with a particular doctor or organization to give the x-rays; or we could 
let the 100 poor people get the x-rays on their own and we would pay 
the bill. 

The important thing is that the objective did not prescribe how we 
should reach our objective. We are free to analyze and select the most 
effective or efficient method available to us. 



Now gather into a small group of 3 to 5 and complete the exercises 
on the following pages. 



Write at least three objectives for each of the first five goals 



prioritized at the last meeting . 
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Goal : 



Objective 1 



Objective 2 



Objective 3 




Goal : 




Objective 


1 




Objective 


2 




Objective 


3 
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Goal : 



Objective 1 



Objective 2 



Objective 3 



Goal: 



Objective 1 



Objective 2 



Objective 3 



Goal 



Objective 1 



Objective 2 



Objective 3 
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Review what you have written and see if the objectives have these three 
qualities: 

1. Is the intended result clear? 

2. Can the intended result be measured? 

3. Does the intended result have a time frame? 

Re-write your objectives if you think they can be improved. 



After you have reviewed your work, assemble together into a large 
group to discuss and agree upon specific objectives for your agency. 



IAR6E_C0MENSUS SHEET - OBJECTIVES 
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Directions for Staff Between Sessions Two and Three 

1. Ask each person (employee or volunteer) who performs activities for 
the agency to list those activities and estimate the amount of time 
he/she puts into those activities. 

2. Have the immediate supervisor of' each person review each list and in 
conjunction with the subordinate make any necessary adjustments so 
that the list is as accurate as possible. 

3. Put a monetary value on the percentage of time spent on specific 
activities by applying that percentage to the staff salary (and 
fringe benefits) involved. 

4. Allocate other direct costs such as equipment and supplies that can 
be identified with particular activities and program elements. 

5. Allocate rent and other fixecj, costs if realistically possible. If the 
realistic division of a cost is not possible, then the initial man- 
power/hours ratio should be utilized in allocating costs. 

6. Develop a tentative program structure and budget based upon the goals 
and objectives established in Sessions One and Two. 
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CHAPTER THREE 
DEVELOPING A PROGRAM STRUCTURE AND BUDGET 

Overview 

Ideally, program budgeting is a close relationship 
between planning and budgeting, with planning as the 
master and budgeting as the servant ... In a planning 
context the budget document becomes a statement of policy 
rather than a grocery list. 

from Werner Z. Hirsch, et. al.. Local 
Government Program Bud get ina; Theory 
and Practice , p. 2. 

A program budget, properly designed, allows you to 
relate expenditures to program objectives. In other words, 
it permits you to calculate easily in a given year how many 
dollars were spent in achieving X units of progress relative 
to goal A. This allows comparisons of efficiency (benefits/ 
costs) from one year to the next and from one prograir> to the 
next. .The ability to calculate rough benefit/cost ratios 
for each program provides a more realistic basis for deciding 
each year whether to reallocate- effort from one program area 
to another. 

A program , as the term is used here, consists of the 
entire set of activities an agency performs that contributes 
to one of its objectives. The first step in structuring a 
program budget is to identify these programs. The second step 
is to calculate the cost of each activity and the overhead 
costs of conducting those activities. 

There is usually some ambiguity involved in developing a 
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program structure. For one thing, the same activity may be 
contributing to more than one . -active. When this occurs, the 
activity may be somewhat arbitrarily assigned to one or the 
other program; or it may be listed as a fraction under each ob- 
jective to which it contributes. In cases where a number of 
activities appear to contribute to several objectives, it is 
realistic and more useful to simply treat these as a separate 
program, the objective of which is to support the other programs 
of the agency. A common example is "administrative support 
services" such as personnel, typing, computer services, or pro- 
gram evaluation. 

Overhead costs (such as rent, utilities, general admini- 
stration) are commonly allocated to each program as a simple 
percentage of all direct costs. However, where there are 
large overhead costs that can be identified with a particular 
program, these should be listed separately under that particular 
objective. A good example of the latter would be the water 
bill for a pool that is used only in the swimming/1 if esaving 
program. 

The rules of thumb in the last two paragraphs are only 
illustrative of the sometimes arbitrary decisions that must be 
made in constructing program budget categories and allocating 
costs to particular programs. The principal criterion to use 
■ in constructing a program budget is its utility in relating 
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costs to performance relative to each objective. In other 
words, it should allow the organization to quantitatively 
assess the return on its investment in each program. 

Full-fledged program budgeting requires not only 
specif ication;^^ of existing programs but also a detailed list- 
ing of alternative means to the agency's objectives. The 
analysis of alternatives should include r -alistic estimates 
of benefits and costs. These estimates may be derived either 
theoretically or from the actual experiences of other agencies 

Critics of program budgeting point out that organiza- 
tions do not have unlimited resources to invest in assessing 
alternative programs. They also have questioned whether # in 
complex organizations, it is possible to produce with reason- 
able effort a program structure that accurately matches activ- 
ities to the appropriate objectives* 

One advantage that a small agency has in introducing 
program budgeting is the relative simplicity of the tasks of 
developing a program structure and assessing alternatives to 
present programs. Recognizing that small agencies have very 
limited planning resources, we have tried to minimize the 
complexity of the program budgeting process as it is presented 
in this workbook v^ile retaining its basic elements and spirit 
of rationality. For instance, we have chosen not to present 
a methodology for developing program alternatives , in the belie 
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that each agency can develop its ovm approach to this task 
in a manner consistant with its needs and capacities. 

This chapter introduces you to the heart of a program 
budgeting process-including the defining of a program struc 
ture and the relating of costs to objectives. The agency 
staff will assist board members in applying these concepts 
to the current year's budget. 



SELECTED REFERENCES 
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A program structure is a hierarchical arrangement of 
goals and objectives vhich depicts the relationships of 
activities, services, and programs to those goals and ob- 
jectives, A program is the set of activities that contributes 
to one of your agency's objectives. 

There are four basic criteria necessary for the design 
of an adequate program structure: 

1. Program objectives, services, and activities should 
be as mutually exclusive as possible. 

2. Activities must relate specifically to particular 
objectives listed in the structure. 

3. All activities of the agency must be included in 
the structure. 

4. The final structure should facilitate the assessment 
and analysis of programs, services and activities 
and their interrelationships. 




What are the four basic criteria for the design of an adequate 
program structure? 

1. ■ 



2. 



3. 



Check your answers on the preceding page. If you answered c ectly, 
go on to the next page. If you missed one or more, study the correct 
answers before proceeding to the next page. 



There are at least four steps you must take in developing a program 
structure. Below, these four steps are listed, but not. in order. See if 
you can put them in the correct order. 

A. Have each person (employee or volunteer) who performs activities 
for the agency list those activities and estimate the amount of 
rime he/she puts into those activities. 

B. Consolidate all the lists and group the activities as to their 
contribution to the achievement of a hierarchy of objectives. 

C. Have the immediate supervisor of each person review each list 
and in conjunction with the subordinate make any necessary 
adjustments so that the list is as accurate as possible. 

D. Identify agency goals and objectives and put them into a 
hierarchical arrangement which depicts their relationships. 

CORRECT ORDER : 
1. 
2. 
3. 
4. 

Check your answers on the next page. 
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Answers ; 

1. D 

2. A 

3. C 

4. B. 

If your answers were correct, you are ready to proceed. If you were 
incorrect, review the correct order and obtain clarification from the 
coordinator if you still do not understand the correct order. 

On the following page is an example of a very simplified version of 
a program structure. Examine it in light of the criteria listed earlier. 
If you have any questions, please ask the coordinator. 
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Simplified Model of Program Structure* 



Objective 

Enable at least Aa. 
50 single parents, 
presently unem- 
ployed and un- 
skilled to work at 
full time employ- 
ment, paying at; 
least the minimum 
wage, wi thin one 
year. 



Service 



Enable at least 50 Aal . 
single parents, pre- 
sently unemployed 
and unskilled, to 
obtain training that 
will enable them to 
obtain full time Aa2. 
employment, paying 
at least the mini- 
mum wage, within Aa3. 
one year. 



Element Activity 



Recruit and Aal .1 

screen Aal .2 

parents. Aal .3 

Aal. 4 " 
Aal. 5 - 

Train parents. Aa2.1 

Aa2.2 

Place parents , AaS.l 
in employment. Aa3.2 

Aa3.3 — 
Aa3.4 — 



Ab. Provide day care Abl . Provide cogni- Abl.l 
services for at tive learning Abl. 2 
least 50 children experience. Abl. 3 
of single parents Abl .4 
who are in train- 
ing programs or in Ab2. Provide social Ab2.1 
full time employ- growth experi- Ab2.2 
ment within the ences. 



next year. 



Ab3. Provide health Ab3.1 
and nutrition Ab3.2 
services Ab3.3 



Ac. Provide transpor- 
tation services for 
all single parents 
and day care child- 
ren to the follow- 
ing places in the 
coming year: 
training sites, 
day care centers, 
medical centers. 



Acl . Schedule rides, 



Ac2. 



Transport 
clients to 
sites. 



Acl 
Acl 



Ac2.1 
Ac2.2 



*This outline is an arbitrary classification scheme. Once all the activities 
have been determined, it might be more logical to include transportation 
services as elements under sub-objectives Aa and Ab. Another level (sub- 
element) may have to be added if this structure does not prove adequate for 
the agency. 
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After you have developed a program structure and have 
refined it so it reflects the current program operations of 
your agency, the next step is to translate that structure 
into a program budget. 

Budgets, traditionally, have been used to insure that 
money is spent in specified expenditure categories (i.e., 
salaries, equipment, supplies). Traditional budgets have 
basically been plans for allocating resources among expendi- 
ture categories ( inputs ) . However, we .live in an age when 
resources are becoming scarce and accountability is the pass- 
word. The traditional budget is not enough. A budget, today, 
must indicate how resources are being allocated among activi- 
ties and services , (outputs) as these contribute to each of 
the organization's objectives. This is what a program budget 
does. 

The traditional budget is organized by categories of expen 
ditures (input) . The program budget is organized by objectives 
under which are listed the activities (outputs) of the agency 
that contribute to each objective. Of course, in a complete 
program budgeting system, both categories must be known. In 
order for the decision maker to know how much his output costs, 
he must know the input. 
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INPUTS OUTPUTS OBJECTIVES 



Solories* 
Rents- 
Equipment « 
Supplies- 



YOUR 
PROGRAM 



Service A 
Service B 
Service C 




Objective 1 



Objective 2 
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What steps are necessary to convert a traditional line 
item budget into a program budget? Once the program structure 
is developed and the activities have been identified, it is 
necessary to take the line item costs and allocate each of 
them to those activities. 

Step 1. The actual percentage of time spent on specific 

activities can be quantified by applying that per- 
centage to the staff salary (and fringe benefits) 
involved. If a staff member spends full time in 
a particular activity, then your task is that much 
easier. Administrati^'e tasks and salaries are much 
more difficult to allocate. 
Step 2. Allocate other direct costs such as equipment and 
■ supplies that can be identified with particular 
program elements. 
Step 3. Allocate rent and other fixed costs if realistically 
possible. If the physical division of a cost is not 
possible, then the initial manpower /hours ratio should 
be utilized in allocating the costs. 
Step 4. Revise and refine the program budget so it includes 
all the line item expenditures and reflects the pro- 
gram structure. 
Now complete the statements on the following page. 
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Completions 

!• Traditionally, budgets have been used to insure that money is spent in 

• 

2. Traditional budgets have basically been plans for allocating resources 
among . • 

3. A program budget indicates how resources are being allocated among 

4. In a complete program budgeting system, both (aP 



and (b) ' must be known. 



5; In order to make a program budget, you must take the (a)_ 



and allocate each of them to the (b) of the program. 



Check your answers on the following page. 
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Answers : 

Specific expenditure categories 
2. ; Inputs' 

3* Outputs or objectives 

4. (a) inputs, (b) outputs 

5. (a) line item (input) costs, (b) activities 

If your answers .were correct, you are ready to proceed. If you 
missed one or more, review pages 60--62 before you proceed. 

On the following page are contrasting examples of a traditional line 
item budget and a program budget. Examine them and make note of the 
differences between them. Examine the program budget in relation to the 
previous example of a program structure. 
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Simplified Model of Traditional and Program Budgets* 



Traditional Budget (Single Parents Program) 



Salaries and Wages 


20,000 


Fringe Benefits 


1,950 


Contract Services 


3/160 


Total Personnel 




Travel 


3,050 


Space Costs 


1 ,000 


'Consumable Supplies 


300 


Equipment 


200 


Miscellaneous 


1,300 


Total Non-Personnel 




Total Costs 





25,110 



5,850 
30,960 



Program Budget (Single Parents Program) 

Training Costs ^ 12,040 

Recruiting 3,010 

Training 7,525 

Placing in Employment 1,505 

Day Care Costs 10,320 

Cogni ti ve Learni ng 3 ,096 

Social Grov/th 2,064 

Health and Nutrition ' 5,160 

Transportation Costs 8,600 

Schedule Rides 2,580 

Transport Clients 6,020 

Total Program Costs 30,960 



*This budget is based on the accompanying limited program structure 
and the figures v;ere chosen at random. The important thing to note 
is the format and relationship the budget should have to the 
program structure* 
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In order to facilitate your involvement in Phase III, the initial 
steps in developing both a program structure- and program budget have been 
accomplished so that you need only complete the final step in the develop 
ment of the program structure and budget. 

Your staff will present to you the results of their initial efforts 
to develop a program structure and budget. It will be up to you to take 
this information and complete the development of a program structure and 
budget. 
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CHAPTER FOUR 
ASSESSING AGENCY PERFORMANCE 
Overview 

Once one has defined the objective that a set of activi- 
ties is directed at achieving, it is possible to judge the 
degree to which a program has succeeded in achieving that 
objective. Program evaluation is the use of scientific methods 
to measure success in these terms. 

A carefully designed program evaluation process will 
allow the staff to measure not only the degree to which the 
program objective is achieved but also a wider range of impacts. 
For instance, careful monitoring of the impacts of a job train- 
ing program would include measurement not only of how many 
clients get what kinds of employment, but also of the training 
program's impact (positive or negative) on clients' psychological 
well-being and relationships with other people. 

There is a wide variety of models for program evaluation. 
The following elements are common to most of them: (1) establish 
ing "baseline" data on the status of incoming clients or target 
popTilations? (2) measurement of or holding constant sources of 
change other than the program itself (the purpose of which is 
to isolate the impact of the program from other causes of change) 
(3) measurem.ent of changes after the program and follow-up on 
clients v^o have left the program to look for delayed effects 
or "relapses." 
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Evaluating the effects of human services programs 
requires consider aJole ingenuity. It is relatively easy to 
measure the success of a garbage collection or street repair 
program; relatively more difficult to determine what constitutes 
"success" in a family counseling, educational, or recreational 
program, is the best measure of family counseling success the 
number of families who stay together or the number of ^bad mar- 
riages ended? Or neither? Is there any reliable measur.e of 
the extent to which a recreational program builds habits of 
cooperation or raises self-esteem? There are answers to these 
questions, but they are as complex as the problems themselves. 

There is no simple answer either to the question of who 
should perform an evaluation. Professional evaluators hired 
from outside are more likely to produce scientifically valid'"'^" 
results that pull no punches. On the other hand, an outsider 
who is unfamiliar with the agency must spend a lot of time 
gaining knowledge already possessed by the agency's staff. 
For small social agencies there may be no alternative to devel- 
oping an internal evaluation capability. 

It is extremely important for an agency to establish valid 
and reliable measures of program impacts that can be executed 
repeatedly. This allows the agency to monitor trends in per- 
formance over time. When ' a basic change is made in the program 
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a baseline data collection should precede the change and a 
second set of measures should follow the change. 

An agency should subject program evaluation efforts 
to the same type of benefit/cost analysis applied to its 
other programs. There is no reason to collect data that do 
not provide information leading to improved decision-making. 
For instance, it is a waste of effort for an agency simply to 
collect detailed information on numbers of people served or 
contactsd. This tells the agency nothing about its impact. 
Evaluation should focus on outcomes rather than outputs . 
Tlrie assumption that provision of service to a client (out- 
put) has a positive impact (outcome) should be treated as an 
hypothesis and subjected to scientific testing. 

This Chapter provides an introduction to such basic pro- 
gram evaluation techniques as indicator construction, research 
design, and data collection. 

Selected References 

Practical Program Evaluation for State and Local Officials . 
The Urban Institute. 

Moursund, Janet P., Evaluation: An Introduction to Research 
Design . Brooks/Cole Publishing Co., 1973 

Hinrichs, Harley H. and Graeme M. Taylor, Systematic Analyses , 
A Primer on Benefit-Cost Analysis and Program Evaluation . 
Goodyear Publishing Co., 1972 

Weiss, Carol H., Evaluation Research/ Methods of Assessing 
Program Ef f ectiveness ♦ Prentice-Hall I'^^Vz. 
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What kinds of data might be collected to measure impact? Here are some examples: 



In Washington, D.C., the effectiveness of trash collection programs is moni- 
tored by having trained observers periodically observe the amount of litter 
left along curbs after collections. 



Result I Some neighborhoods are found to have received far better service 



than others. Reforms are proposed. 



j A 1970 evaluation of court procedures for placing neglected children shows 
that many youngsters who should have been sent home iirmediately are being 
institutionalized for several weeks before the court can act. 

Result I A proper servicing program is instituted to identify such 



children and provide them with emergency services in their homes .rather than 
place them in institutions. 



An evaluation of recreational programs in one city leads to these discoveries 

1. Citizens in poorer neighborhoods rate their recreation services inade- 
quate and see a need for more organized programs, especially on weekends. 

2. Certain types of equipment are responsible for 80% of all serious 
accidents in park areas. 

3. Accessibility to facilities and programs varies widely and is frequently 
denied to the aged, handicapped^ and those who work certain shifts. 
["Result I Several programs are rescheduled and decentralized. Plans are 

made to replace dangerous equipment. The basic thrust of the recreational 
program in poorer neighborhoods is altered and placed under review by a 
citizens advisory board. 



What do these examples show? Evaluations provide the basis for wise decisions 
by the board and staff based on firm evidence of program impact . 

7 7 
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What are the steps in setting up an evaluation process? 

First, for each objective, list as many types of evidence ( indicators ) as 
you can think of. 

For instance, to measure the success of a drug abuse control program, you 
might need data on the following: 

1- Percent of persons arrested in the jurisdiction who are addicts 
or are under drug influence. 

2. Number of drug overdose deaths. 

3. Percent of those entering your program as addicts who remain free 
of drug use for one, two or three years after undergoing treatment. 

4. Percent of those entering your program who were unemployed but have 
held steady employment for at least a year or returned to school. 

5. Percent of those entering your program with criminal convictions 
who have not been charged with crimes for one year, two years, or 
longer after undergoing treatment. 

6. Improvements in mental and physical health of those undergoing 
treatment. 

7. Percent who relapse and return to the programs. 

8. Program retention rates and reasons for dropping out. 



Data on these eight indicators ought to be collected on a regular basis. 
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Now take one of the objectives of your own agency's programs and list as 
many measurable indicators of impact as you can think of: 



Objective: 



Indicators: 
1. 



3. 
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What is a good indicator? 

1. It must be valid . That is, it must be a measure of 
impact relative to your objective. 

2. It must be reliable . (A sundial won't tell time on 
cloudy days. It's unreliable ) . 

3. It must not be too expensive to collect. 

4. Collection of data must not violate constitutionally 
protected rights or unnecessarily deny someone his 
privacy. 

5. Data should be collected as regularly and frequently 
as necessary. 
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What are the most comnonly used techniques to gather data on performance? 

A. Agency records : Data that is presently collected can often be looked 
at in a wholly new light in order to assess program performance. 

2. Other public records : Especially useful in following up on previous 
clients and measuring the success of referral programs. 

3. Surveys : Present or past clients may be interviewed. Also, a rela- 
tively small, scientifically drawn sample can provide a reliable 
picture of how citizens of a neighborhood or city have felt the 
impact of your program. 

4. Experimental Designs : This requires use of a control group that is 
not going through your usual program. Measures are taken before and 
after the program for both groups and the results compared to discover 
any impact. 

5. Census Data : Often useful in making decisions about the location of 
a program site or other decisions involving geography. 
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Where possible, evaluators should use several indicators for each objective. 

Evaluators should also look out for negative impacts-- undesirable side 
effects . 

It is up to the board members, each year to weigh: 

1 . The positive impacts of each program. 

2. The negative impacts of. each program, and 

3. The cost of each program. 

And then, on the basis of valid , reliable data for all three, decide whether 
each program should be expanded , reduced , changed , or eliminate d. 
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Now, let's take a few minutes to work in groups on the construction of 
indicators* 

Each group should consider indicators for one of your agency's objectives. 
For each indicator, ask two questions: 

1. Is this a valid , reliable indicator? 

2. Do we have, or is it practical to collect, the data necessary 
to employ this indicator? 
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.A Some Propositions for Discussion 




1. Our agency doesn*t need to research its target population, 
because we already know who is eligible and who needs our 
services. 

2. Our agency doesn't need to gather more detailed information 
on incoming clients since we are already adequately diag-- 
nosing their needs and the causes of their problems. 

3. Our agency *s programs can^t be evaluated because there is 
no way to measure success. 

4. Clients are poor judges of the quality of service they are 
receiving. 

5. We can*t afford what it would cost to do a proper job of 
.evaluation in this agency. 
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Chapter Five 
DEVELOPING NEXT YEARNS PROGRAMS 
Overview 

At the completion of Chapter Four, you will have accomplished the 
fol lowing: 

1. Agreed on goals and priorities. 

2. Learned to write specific goal-related objectives. 

3. Developed. a program structure and budget based on current 
agency acti vi ties. 

4. Assessed agency performance levels and made reconmendations 
for improvement. 

Now you are prepared to start planning next year's programs in light 
of the information already developed and in light of new information to be 
developed. This chapter will provide you with a guide to the things that 
need to be done in order to keep this yearly planning-programming-budgeting 
process on schedule. 

Gui^lelines are provided on (1) how to assess conmunity problems, and 
(2) how to develop and analyze alternative solutions to those problems. 
These two processes, added to what you already know about your current goals, 
objectives and activities, will assist you in refining your programs so they 
win be more relevant and more effective next year. The annual PPBS Calendar 
at the end of the chapter should be posted as a quick reference to where 
you are in the process at any time during the year. 
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A Guide to Assessing Community Needs and Problems 
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An effective job of establishing long range goals and developing 
strategies to accomplish your agency's desired ends cannot be done without 
valid and reliable information about the environment in which you operate. 
In fact, your agency's success will probably be in direct proportion to the 
amount of specific information you are able to gather about your community's 
needs and problems. 

Most agencies' information begins with the experiences of board and 
staff members. However r Tood 'decisions are seldom based on intuitive 
judgments alone . It may be necessary to base initial planning activities 
on such judgments, as we did in Chapter One, but as the planning process 
proceeds, the need for more detailed information will become apparent. 

Types of information that usually help describe a community and its 
problems include: 

1. Demographic characteristics such as population size, composition 
and distribution 

2. Economic characteristics such as descriptions of industry, 
business climate and labor force 

3. Social structure characteristics and influence of legal, poli- 
tical, cultural, educational and religious organizations 

4. Physical and geographic characteristics such as natural elements, 
housing patterns, and movement patterns 

Some sources of these types of information, in addition to the board 
and staff, include: - 

1. Special studies by planning agencies or universities 

2. Mailed questionnaires 

3. Personal or telephone interviews (preferred over mailed questionnaires) 
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4. Data generated by existing social service agencies 

5. Census data and other government reports 

6. Local government offices such as welfare, housing authorities, 
transportation agencies or zoning offices 

7. Local utility companies 

8. Local school boards 

9. Local chamber of conmerce 

10. Newspaper files 

11. Libraries 

12. Connunity meetings to hear what citizens want "ind perceive to 
be their needs 

The purpose of gathering information is to lay the data base for 
analysis. Analysis should ask at least the following questions: 
What is the real social problem being addressed by the agency? 
Why is it a problem? 
Who is affected by the problem? 
What is the magnitude and severity of the problem? 
Where is the problem? 

What are the major causes and side effects of the problem? 
il must be emphasized that information al one will not identify a^ Problem . 
Analysis of the^ information is necessary before problems can be identi fled . 
This is usually a multi-stage activity involving initial analysis using 
limited information which in turn identifies the need for even more specific 
types of information. For example, a problem statement which says, "Many 
people in our cormiunity need child day care services," is inadequate and 
demonstrates the need for additional information. "How many people need 
what kinds of day care service?" "Who are these people and where are they 
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located?" "Why do they need this service?" This level of specification 
begins to approach a point where a precise problem statement can be formu- 
lated and upon which goals and objectives can be established. 

Some may initially feel that problem statements are easy to make and 
that generally everyone will agree as to what constitutes the problem. 
NotMng could be farther from the truth. The most time-consuming portion 
of the planning process is in problem identification. It has been said that 
as much as 50 percent of the effort to solve a problem should be applied to 
thinking about the problem. It is more important to find the true problem 
than it is to follow a perfect analytical procedure to the solution. Peter 
Drucker, noted management consultant, once said "... the important and 
difficult job is never to find the right answer, it is to find the right 
question. For there are few things as useless-if not as dangerous-as the 
right answer to the wrong question." 

It is necessary to insure that problem statements actually depict the 
problem and not a symptom of the problem. For example, the statement that 
day care services are needed in the community does not go to the root of the 
situation. The problem, more specifically, may be that there are single 
parent families wherein the breadwinner cannot work and take care of the 
children at the same time. There may be alternatives to the solution of „ 
this problem other than just day care services. The first problem statement 
limits the alternatives to be considered by the decision maker to types- -of 
day care services. The restatement of the problem expands the decision 
maker's field of vision. Day care may or may not be the best and most 
economical way of resolving the actual problem, but now the decision maker 
can consider other alternatives and compare their cost or effectiveness to 
day care in resolving the basic problem. 
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The goal setting process should flow smoothly and logically from your 
problem analysis and problem statement. But before your board determines 

what problems and goals should receive top priority, it should receive 

i- 

input from the comnunity. Just as information is a prerequisite to developing 
problem statements, so alsq„the goal setting process has a prerequisite,— 
community involvement . In order to insure that the goals selected have a 
chance for accomplishment, it is advisable that as many as possible of the 
following groups be included in the goal setting process: target population, 
power groups, professional groups, related comnunity organizations, volunteers 
board members, and staff members. 

How is it possible to get all of these groups to agree on what goals 
should be pursued? The only way to do this is to establish an open ongoing 
dialogue with all of the groups concerned. There are people at the University 
who can assist you in establishing such a process. If you feel you need 
help in this area contact the project coordinator (745-8494), or the 
Youngstown State University Center for Urban Studies (745-1851 , Ext. 498). • 
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A Guide to Developing and Analyzing Program Alternatives 

Once you have completed the problem analysis and goal setting stages, 
you must determine the most effective or efficient method available to you 
in reaching a particular goal. 

In-order to examine or analyze any program alternative, it is essential 
that a model be developed describing the relationship between resources used, 
how they are used, and the results achieved or expected. Your ability to 
describe the relationship will determine the validity of your comparison of 
alternatives. 

Information about the effect of strategies must be available to you if 
you are going to compare program alternatives. The closest source of this 
information is your own agency. If you have evaluated what your agency is 
doing, how it is doing it, and the impact it is having, then you have some 
information about the effect of your current strategies. Other sources of 
information on the effect of strategies can include specialists who are 
familiar with various courses of action for a particular problem or reports 
of the experiences of other agencies with problem situations similar to yours. 

Some of the major difficulties and limitations in analyzing and 
implementing alternatives include: 

1. Lack of specifically measurable and demonstrable objectives 

2. Inadequacies in data and knowledge, such as not knowing the 
effects of particular alternatives nor even being able to 
describe the current situation 

3. Inability to accurately predict the future 

4. Statutory or fiscal restrictions outside your control 

5. Narrow perspectives and experiences of decision makers 
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Do not become discouraged if these obstacles sometimes appear insur- 
mountable. In any analysis, we may often have to be satisfied with using 
the process without arriving at final .answers. Partial analysis is better 
than no analysis. The depth and sophistication of analysis is not as 
important as bringing assumptions and uncertainties into the open. It may 
be that the greatest initial benefit of attempting to analyze alternatives 
is the dialogue opened up between board, staff and comnunity. 
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Program Budgeting Calendar 

The calendar on the following page indicates what must 
be done each yeair to maintain a program budgeting process that 
will increase your agency's effectiveness. The dates shown 
are estimates, but you should make every effort to include all 
steps in the process in the right order. 
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Completion 
Date 

Ongoing 
Feb 28 
Mar 15 
Mar 31 
May 15 
jun 30 
Jul 3i 
Aug 31 
Sep 15 
Oct 1 
Nov 15 
Dec 10 
Dec 31 
Jan 1 
Ongoing 
Periodic 
Periodic 
Feb 28 



Program Budgeting Calendar 
Staff Action Board Action 



Data Collect-ion 



Problem definition^ ^Problem definition 



Establish priorities 



Develop a 



jPevelop specific objectives 



ternati ves 



Analyze al ternati ves] < » |Analyze alternative! 



Select a 



ternati ves 



Write program plain 



>| Approve program plan 



(Tubmit p. 



an for funding 



- {Accept fundi ngt 4 



Implement planj 



Monitor programs 



T ^Evaluate program 



Recommendations for 
program changes 



Restart cycle by 
reexamining problems 
and priorities 


< > 


Restart cycle by 
reexamining problems 
and priorities 
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Chest Action 



Review plan 



Approve funding 
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